An LSPI based reinforcement learning approach to enable network cooperation in cognitive wireless sensor networks

The number of wirelessly communicating devices increases every day, along with the number of communication standards and technologies that they use to exchange data. A relatively new form of research is trying to find a way to make all these co-located devices not only capable of detecting each other's presence, but to go one step further - to make them cooperate. One recently proposed way to tackle this problem is to engage into cooperation by activating 'network services' (such as internet sharing, interference avoidance, etc.) that offer benefits for other co-located networks. This approach reduces the problem to the following research topic: how to determine which network services would be beneficial for all the cooperating networks. In this paper we analyze and propose a conceptual solution for this problem using the reinforcement learning technique known as the Least Square Policy Iteration (LSPI). The proposes solution uses a self-learning entity that negotiates between different independent and co-located networks. First, the reasoning entity uses self-learning techniques to determine which service configuration should be used to optimize the network performance of each single network. Afterwards, this performance is used as a reference point and LSPI is used to deduce if cooperating with other co-located networks can lead to even further performance improvements

Traffic differentiation: a basic step towards providing end-to-end QoS on the train-to-wayside wireless communication system

We developed a network platform that is responsible for an uninterrupted and seamless connectivity from the train to the wayside through heterogeneous wireless access technologies. However, limiting the offered services to only an onboard Internet is not a feasible business case. A viable one should extend to a broad spectrum of railway communication services like: train control, diagnostics, real time passenger information, entertainment, security CCTV surveillance etc. In a highly dynamic environment (from the communication link point of view) such a fast moving train, it is neccesary to introduce prioritization among different traffic classes. This will implicitly determine under what conditions a certain flow should get suspended or dropped in order to preserve the flows of a higher priority as long as possible and to ensure that they meet their QoS demands. The first step towards this goal is data traffic differentiation

Experimental validation of a reinforcement learning based approach for a service-wise optimisation of heterogeneous wireless sensor networks

Due to their constrained nature, wireless sensor networks (WSNs) are often optimised for a specific application domain, for example by designing a custom medium access control protocol. However, when several WSNs are located in close proximity to one another, the performance of the individual networks can be negatively affected as a result of unexpected protocol interactions. The performance impact of this 'protocol interference' depends on the exact set of protocols and (network) services used. This paper therefore proposes an optimisation approach that uses self-learning techniques to automatically learn the optimal combination of services and/or protocols in each individual network. We introduce tools capable of discovering this optimal set of services and protocols for any given set of co-located heterogeneous sensor networks. These tools eliminate the need for manual reconfiguration while only requiring minimal a priori knowledge about the network. A continuous re-evaluation of the decision process provides resilience to volatile networking conditions in case of highly dynamic environments. The methodology is experimentally evaluated in a large scale testbed using both single- and multihop scenarios, showing a clear decrease in end-to-end delay and an increase in reliability of almost 25 %

Design and prototype of a train-to-wayside communication architecture

Telecommunication has become very important in modern society and seems to be almost omnipresent, making daily life easier, more pleasant and connecting people everywhere. It does not only connect people, but also machines, enhancing the efficiency of automated tasks and monitoring automated processes. In this context the IBBT (Interdisciplinary Institute for BroadBand Technology) project TRACK (TRain Applications over an advanced Communication networK), sets the definition and prototyping of an end-to-end train-to-wayside communication architecture as one of the main research goals. The architecture provides networking capabilities for train monitoring, personnel applications and passenger Internet services. In the context of the project a prototype framework was developed to give a complete functioning demonstrator. Every aspect: tunneling and mobility, performance enhancements, and priority and quality of service were taken into consideration. In contrast to other research in this area, which has given mostly high-level overviews, TRACK resulted in a detailed architecture with all different elements present